System and method for locally generating data

ABSTRACT

A method provides pre-rendered information messages for display by a peripheral display unit connected to a base station over a data link. The method involves detecting that a data connection between the base station and the peripheral display unit is available, sending pre-rendered information messages from the base station to the peripheral display unit, and storing the pre-rendered information messages at the peripheral display unit for display in response to a status detected at the peripheral display unit. The base station may perform the pre-rendering of information messages in response to variation in configuration information relating to the peripheral display unit. The pre-rendered information messages may be paired with peripheral display unit statuses and the peripheral display unit may be configured so that, in the event that a particular status is detected, the corresponding pre-rendered information message is displayed without requiring data transmission from the base station.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national phase application of InternationalApplication No PCT/GB2018/052832, filed Oct. 4, 2018, which claimspriority to Great Britain Patent Application Serial No. 1716329.6, filedOct. 5, 2017, all of which are incorporated herein by reference.

BACKGROUND

This invention relates to the generation of data for display byperipheral display devices, and in particular to the generation of datafor display at times when an external connection to the peripheraldisplay device is not available.

In modern audio-visual display systems, it is common to separate thedisplay elements from the image processing, generation and renderingportions. This separation allows the potentially extremelycomputationally complex parts of the process to be carried out at a basestation (possibly a purpose built one), while the display parts can beoffloaded to a peripheral unit. The peripheral unit can be lighter andcheaper thereby making such devices more portable and reducing theircost since they can be configured to unintelligently display the contentwhich is provided to them. The content can be supplied from the basestation to the peripheral display unit by a suitable connection, forexample, cables, optical fibres, wireless communications, etc. Specificexamples of peripheral display devices which fall into this category areprojectors and virtual reality (VR) headsets or augmented reality (AR)headsets.

AR and VR devices in particular have large computational loads, as anentire 3D environment must often be generated in real time for displayto a user. There is a complex feedback loop between: detected movementof a user's head; generation of a 3D environment; generation of a pairof high quality video streams (one for each eye), correctly rendered fordisplay on the particular headset, with the current settings; anddisplay of the two streams by the headset. Space and weight constraintsat the headset mean that the only feasible division of labour betweenthe base station and the headset is to have the base station take on thevast majority of the generating, rendering, motion tracking, etc. and toleave the headset to simply display what it is sent.

A further complication is the current drive towards wirelessconnectivity. While a convenient development in most cases, in thesituation described above it can be problematic to try to send videodata over a wireless link, since the usable bandwidth of wireless linksis known to fluctuate. Fluctuations in bandwidth available to aperipheral device may be further worsened where such peripheral devicesare repositioned, as is often desirable, particularly in AR and VRsystems where movement of the user (and correspondingly the AR or VRheadset) is often necessary in order to gain a full appreciation of thevisual data being displayed.

Sending video content over wireless links can be particularlyproblematic with respect to reliability of the link, due to therelatively large bandwidths required for transmission of even moderatequality video. For example, an uncompressed 1080p video with a colourdepth of 24 bits, being transmitted at a framerate of 60 fps requiresnearly 3 Gbit/s. While compression can help alleviate this burden, it isclear that bandwidth fluctuations can cause problems in the smoothtransmission of video. In extreme cases, the movement of the device cancause the bandwidth to drop so low that no video can be transmitted atall (or at least not in a reasonable time). Typically, in such cases aseries of standard error messages may be stored in memory associatedwith the peripheral display unit, and shown when a corresponding erroris shown. However, as the complexity of peripheral display devicesincreases, so too do the calculations which need to be performed inorder to correctly display images and/or video on them. In particular,each of projectors, AR headsets and VR headsets often have complicatedoptical systems. These optical systems require substantial rendering tobe applied to raw image data to correctly display the image data withoutdistortion effects, some of which may change over time, according touser preference. Moreover, there may be other parameters associated withsuch error messages, such as the language in which to present theinformation. It is a problem for designers of such systems to ensurethat any necessary error messages can be displayed by the peripheraldisplay unit. The number of different combinations of languages,focusing parameters, etc. is simply too much to store on a peripheraldisplay unit, in which the general drive is towards simplification.Nonetheless, when the data link between the base station and theperipheral display unit is unreliable, the peripheral display unit canend up in a situation in which a particular status to which a usershould be alerted is detected, but it is only able to display aninappropriate (e.g. badly rendered, incorrect, etc.) message. In thiscase, it can either display the inappropriate message or a blank screen,in the hope that the user will be prompted to try to resolve the issuethemselves.

The problem is particularly acute in VR systems (and to an extent in ARsystems), because the experience is so immersive, meaning that suddenchanges in rendering style, quality, focus or indeed the very imagesdisplayed (e.g. transitions to a blank screen, or rapidly changingcolours) can all result in disorientation, discomfort and nausea. In ARand VR too, the simple solution of using a cable to connect the basestation to the peripheral display unit is inappropriate as movement isoften required. Moreover, since AR and VR systems tend to be wearable(e.g. headsets), increasing either the storage capacity or computationalcapacity of the headset is not a practical solution either, due toweight constraints.

SUMMARY

The present invention, as defined by the appended independent claimswith preferred features being presented in the dependent claims, aims toaddress some or all of the above-mentioned problems.

Disclosed herein is a method of providing pre-rendered informationmessages for display by a peripheral display unit when the peripheraldisplay unit is connected to a base station over a data link, the methodcomprising: detecting that a data connection between the base stationand the peripheral display unit is available; sending pre-renderedinformation messages from the base station to the peripheral displayunit; and storing the pre-rendered information messages at theperipheral display unit, for display in response to a status detected atthe peripheral display unit. This arrangement allows the peripheraldisplay unit to obtain pre-rendered information messages for display toa user when a connection is available. The peripheral display unit canthen display these images to a user, whether or not the peripheraldisplay unit is able to communicate with the base station. This solutionleverages the existing powerful video processing systems to provide therequired information messages, correctly rendered for the currentdisplay settings and parameters, to the peripheral display unit at timeswhen sufficient signal strength is available to do so. This allows animproved display quality of such messages. In this context a basestation may be the entirety of the generating, processing, compressing,rendering and transmitting parts of the system, or it may comprise onlysome of these parts. For example, in cases where the rendering happensat the base unit, in response to variation in configuration informationrelating to the peripheral display unit, there need be fewer componentscommunicating with one another to provide the correctly renderedcontent, and consequently less reliance on transmission acrosspotentially unreliable links. Alternatively, the messages could beprovided in a pre-rendered format to the base station. In this scenario,the messages could be sourced from e.g. an internet source such as aserver managed by the manufacturer of the headset, and/or the basestation and/or a developer of software for these portions of the system.The messages could be provided from such a source to ensure that themost recent variants of the messages are provided to the system.

There are several effects which may require correction by apre-rendering step, some due to user settings or abilities, while othersare due to parameters of the display unit. Mura artefacts are caused byvariations in brightness between adjacent pixels, even if they aresupposed to be outputting the same brightness, and are caused by randomfluctuations in the process of producing display pixels. Renderingsettings can be adjusted to minimise the obviousness of this effect to aviewer. The lens system may introduce effects due to chromaticaberration or other focal parameters of the lens system. This can bepre-corrected for by providing a distorted image on the screens, asdescribed in more detail below. In addition to the lens system causingdistortions, the display panels themselves may be curved or angledrelative to one another or to the sight axis of a viewer. This effect,particularly when filtered through a lens system as set out above, canrequire additional pre-rendering to provide the desired images to aviewer. Indeed, even flat lenses may be positioned a variable distancefrom the user's eyes (e.g. to improve user comfort), which may requirethe rendering parameters to be varied to account for this. Depending onthe design of the headset, ambient lighting may be taken into account,for example to increase brightness output by the panels if the ambientlighting is bright, so as to ensure that the images displayed arevisible.

In addition, the user may have various accessibility requirements whichaffect how messages are to be displayed. For example, the user may havevisual impairment, colour blindness, astigmatism, issues with the retinaor other eye parts, macular degeneration, etc., all of which may requireadjustments to the arrangement of images or image colours to bedisplayed, and which can be at least partly corrected for bypre-rendering accordingly. In some cases, the pre-rendering may includeenhancing brightness or saturation towards the centre of the displayedimage. In some cases, it can be helpful to think of the lens system asextending to include also prescription eyewear (glasses, contact lenses,etc.) that a user may be wearing. The user may adjust the displayparameters (and therefore the rendering parameters) to account for this.In some cases, the user may make an adjustment (with correspondingrendering parameter adjustments) to adapt the headgear to theirinter-pupillary distance.

There may be different rendering settings based on circumstances or userinformation. For example, information may be modified before beingdisplayed to a child or adult with learning difficulties. In anotherexample, where the display unit is being used to train a user (e.g. incounter terrorism, paramedic, firefighting, etc.), a flag may be set topresent slightly different information in the event of an error, toensure that the user is aware that the error message is not part of thetraining or simulation, but relates to a real-world event.

Additional parameters to take into account for rendering steps are onesrelated to user activity. For example, if it is known that the currentdisplay data is likely to require a large amount of movement from theuser, it may be desirable to alter how messages are displayed, e.g. butcausing them to slowly fade into view, since rapid changes in perceivedmovement (especially in VR situations) can cause nausea anddisorientation. In some cases, a user may lose their balance if there isa rapid transition from fast movement to no movement. Likewise, if thedata being presented is typically very dark, a bright message could bejarring and painful to a user. This could be predicted and adjusted for,for example by providing a dimmer message image the than the usualbrightness. In some circumstances a very bright average data may resultin a brighter than average message being displayed, for similar reasons.General information about the typical scene likely to be encountered maybe provided, for example by considering an entropic measure of the“busyness” of the background.

Optionally, the pre-rendered information messages are stored in anon-volatile memory at the peripheral display unit. The use ofnon-volatile memory allows the correctly rendered messages to be storedin the memory of the peripheral display unit in such a way that shuttingdown the peripheral display unit will not erase the correctly renderedmessages, nor will accidental or deliberate powering down of theperipheral display unit.

In some examples, the messages are stored with a priority. This prioritycan be used, for example to determine which messages are stored innon-volatile memory. For example, the messages which are most likely tobe needed before a connection has been established with the basestation, or most likely to be needed when the connection fails, could bestored in the non-volatile memory. Other messages are more likely to beable to be sought prior to being needed, so need not be stored innon-volatile memory. Other ranking systems could be used to determinewhere to store messages, such as how common those messages are.

In some cases the pre-rendered information messages are paired withperipheral display unit statuses and the peripheral display unit isconfigured so that, in the event that a particular status is detected,the corresponding pre-rendered information message is displayed withoutrequiring data transmission from the base station. This allows a user tobe correctly alerted to a wide variety of statuses even when there is noconnection available. Examples of such statuses and messages include:

-   -   No connection is available. This is a key case where it is        important that the message be provided in advance, since it will        not be possible to render this message correctly when the error        occurs.    -   Low bandwidth. For similar reasons as set out above, this is        important to have in advance.    -   No data detected. In the case where the connection is (or        appears to be) working correctly, but data is nonetheless not        being received, it will be impossible to render messages to        alert a user to this.    -   Base station and display unit are incompatible. In some cases,        it may be that these two parts of the system cannot communicate        with one another adequately to render the data required of them.        They may nevertheless have an emergency set of parameters which        can be used to provide a readable message to a user to alert        them to the problem.    -   System damage. Part or all of the system may suffer an error        (e.g. due to software updates or physical damage) which prevents        further communication.    -   Antennae misaligned. In this case, the connection quality may be        reduced (resulting in a specific case of the above errors).    -   Proximity/out of bounds. The user may be reaching the edge of        the usable region (in the real world), either due to        communication (e.g. line of sight) limitations or due to        physical space limitations (the room they are in is too small).    -   Overheating. One or more parts of the system may be about to        shut down or otherwise fail.    -   Fire/system alarm. Alert the user to events outside of the        virtual world.    -   Headphones not present.    -   Microphone muted.    -   Audio in/out interrupted.    -   Session expired/insert more money or tokens/unauthorised access.        For pay per hour systems etc. there may be a need to alert a        user to the situation when a host system is configured to shut        down after a certain time, etc.    -   Host system restarting/updating/busy. Where the host is required        to restart, update, or generally be busy doing something else,        the user should be able to be alerted, since the host may be        unresponsive at such times.    -   Power system failure/critical.    -   Fingerprint/voice recognition request. In case a user finds        themselves locked out of the host.    -   Too many devices on the same radio channel (channel congestion).        This would be a special case of the low bandwidth case above.

In any of the above cases, the message may include suggestions such as“move closer to base station” or “check for line-of-sight obstacles” totry to remedy the situation, in addition to alerting the user to thesituation. Audio channels may be used to back up or add to the messagessent. Audio streams could be sampled and sent to the device to be storedfor the above scenarios too. This could be used in situations where theimage should not be interrupted, or if the user has accessibilityissues, or if the display panel is reporting problems/is missing, or theevent is particularly critical, e.g. fire alarm.

The pre-rendered information messages may be error messages and thestatuses may be error statuses. This is particularly important as manyerrors may make it impossible for the peripheral display unit to contactthe base station (or indeed anything else). Consequently, it isadvantageous to prepare in advance for such error statuses bypre-loading error messages to prompt a user to take corrective action.In particular when there is no signal available due to the unreliablelink, it is important to be able to inform a user of this, so that theycan move the peripheral display unit in an attempt to re-establish theconnection. It is important that such a message is correctly rendered asin extreme cases it may be impossible to read a poorly rendered message,leaving the user in no better of a situation than a blank screen wouldhave. Situations may even arise in which the peripheral display unitdoes not know what the solution is. In these cases, it may beadvantageous to have a default error message to revert to. This may bean instruction to move the display unit closer to the base station in anattempt to improve the communications link, so that the base station canbe interrogated (possibly even utilising a connection to the internet tohelp resolve the issue), for example.

In some cases the peripheral display unit is configured to sendinformation messages to be rendered to the base unit once availabilityof a connection between the peripheral display unit and the base stationis detected. This allows a peripheral display unit to be provided with aset of messages which might be useful to it, i.e. to cover the varioussituations envisaged by the manufacturer. When a communications link isavailable, the peripheral display unit can send these messages to thebase station which performs operations on the basic images to renderthem for display by the peripheral display unit, before sending therendered images back to the peripheral display unit.

Additionally or alternatively, the base station may be configured todetermine messages for sending to the peripheral display unit. Thisallows the base station to provide the peripheral display unit withadditional information for display, which may not have been available tothe peripheral display unit at the time it was manufactured. Theperipheral display unit may be given the opportunity to accept or rejectmessage proposals. In the event that the messages are to be sent to theperipheral display unit, the rendering of the messages can occur in thesame manner as that set out above. The proposed messages may be pairedwith status conditions, as described above. In some cases, this pairingmay include additional, non-visible information for the peripheraldisplay unit, to tell it how to determine when the corresponding statushas occurred, for example.

The pre-rendering may take account of one or more of: language settingsof the base unit and/or the peripheral display unit; chromaticaberration of the peripheral display unit optics; focal parameters ofthe peripheral display unit optics; brightness variation in pixels (e.g.mura artefacts); other display parameters of the peripheral displayunit; and/or one or more parameters of image or video data currentlybeing displayed by the peripheral display unit. Each of these isimportant for ensuring that the message can be seen and correctlyinterpreted by a user. For example if a user only speaks English, thenmessages in Chinese will be of no use to them. A user may have set theirlanguage preferences on the base station, so there is no way that theperipheral display unit would know which language to present themessages in without communicating with the base station. Similarly,different peripheral display units each of which are compatible with thebase station may nonetheless require different rendering to account fordifferences in their optical systems and/or user preferences. When theperipheral display unit connects to the base station, part of thehandshaking and verification stage can include the peripheral displayunit transmitting some of these parameters to the base station to allowthe base station to correctly render the images or video. Lastly, it ispossible that the information message to be overlaid on the display mayneed to be adapted to the other video data being sent. For example, inorder to read text on a background, the text should contrast with thebackground, so the message rendering may take into account average pixelcolours, most common pixel colours (or colour ranges) averagebrightness, etc. to maximise comfort or minimise discomfort from jarringcolour combinations or brightness transitions. In addition, thecorrections required to correctly display information to a user toaccount for curved or angled panels, distance of a display panel from aviewer's eyes, ambient light levels, accessibility requirements,inter-pupillary distance, user or circumstantial factors, and/or motionor location of a user.

The method described above may be executed every time a connection isavailable. This ensures that the most up to date settings are used.

Additionally or alternatively, pre-rendered information messages aresent in response to the base station and/or the peripheral display unitbeing initialised. In other words, when the base station and/orperipheral display unit is switched on and loaded up (i.e. booted),there may be additional messages sent to supplement the existingmessages or replace the entire set. This provides a good periodic updateto the system, and also allows the message display settings to be resetto default values in the event that any display parameters of the systemare also reset to their defaults.

Additionally or alternatively, the method may be executed after softwarefor the base station and/or the peripheral display unit has beenupdated. This allows the messages to be correctly rendered in the eventthat any relevant settings are changed in a software update. This isparticularly important as error statuses may sometimes be more likelyjust after a software update.

Additionally or alternatively, the method may be executed after alanguage setting for the base station and/or the peripheral display unithas been changed. As discussed above, the language in which a message ispresented can be critical to prompting a user to take the desiredaction. Similarly, the method may be executed after a graphical displaysetting has been changed. As set out above, the graphical displaysettings can be key to getting the message to display in the desiredmanner to allow it to be correctly read by a user. In AR and VR, this isparticularly important, since out of focus or otherwise poorly renderedimages are known to cause discomfort and nausea in users.

Additionally or alternatively, the pre-rendered information messagesstored on the peripheral display unit may be updated when a change inone or more parameters of image or video data currently being displayedby the peripheral display unit is detected. The parameters may relate toaverage brightness or colour parameters, for example. This can allowmessages to be provided with the most up to date information about themakeup of a typical visual frame. Consequently, an appropriate level ofcontrast and non-jarring colour and brightness combinations can beensured.

Additionally or alternatively the method may be executed in response todetection of a particular status by the base station and/or theperipheral display unit. This allows, for example, the peripheraldisplay unit to detect that it has accidentally deleted or overwrittenone or all of the messages, and to request that it be sent replacements.

Additionally or alternatively, the method may be executed periodically.This can ensure that the messages are kept up to date, even if none ofthe other triggering events discussed above occur.

In each of the above examples in which the method is repeated based onsome trigger, the new pre-rendered messages which are sent when themethod is repeated will also be stored in the peripheral display unitmemory. In some cases, these will overwrite the previous equivalentmessages. In other cases, they may be stored in addition to the existingmessages. Clearly, this latter case will eventually fill the availablestorage space. Consequently, the peripheral display unit may delete theoldest messages to make way for the new. Alternatively it may delete theleast used messages to make space for the new messages.

Also disclosed herein is a system for providing pre-rendered informationmessages for display by a peripheral display unit when the base stationand the peripheral display unit are connected to one another over a datalink, the system comprising: a base station; a peripheral display unithaving a memory; and a data link between the a base station and the aperipheral display unit; and wherein in the event that the peripheraldisplay unit and/or the base station detect(s) that a connection isavailable between the base station and the peripheral display unit isavailable, the system is configured to trigger sending of pre-renderedinformation messages from the base station to the peripheral displayunit; and wherein the peripheral display unit is configured to store thepre-rendered information messages in the memory, and to display thepre-rendered information messages in response to a status detected atthe peripheral display unit. This arrangement allows the peripheraldisplay unit to obtain pre-rendered information messages for display toa user when a connection is available. The peripheral display unit canthen display these images to a user, whether or not the peripheraldisplay unit is able to communicate with the base station. This solutionleverages the existing powerful video processing systems to provide therequired information messages, correctly rendered for the currentdisplay settings and parameters, to the peripheral display unit at timeswhen sufficient signal strength is available to do so. This allows animproved display quality of such messages. In this context a basestation may be the entirety of the generating, processing, compressing,rendering and transmitting parts of the system, or it may comprise onlysome of these parts. For example, in cases where the rendering happensat the base unit, in response to variation in configuration informationrelating to the peripheral display unit, there need be fewer componentscommunicating with one another to provide the correctly renderedcontent, and consequently less reliance on transmission acrosspotentially unreliable links. Alternatively, the messages could beprovided in a pre-rendered format to the base station. In this scenario,the messages could be sourced from e.g. an internet source such as aserver managed by the manufacturer of the headset, and/or the basestation and/or a developer of software for these portions of the system.The messages could be provided from such a source to ensure that themost recent variants of the messages are provided to the system.

There are several effects which may require correction by apre-rendering step, some due to user settings or abilities, while othersare due to parameters of the display unit. Mura artefacts are caused byvariations in brightness between adjacent pixels, even if they aresupposed to be outputting the same brightness, and are caused by randomfluctuations in the process of producing display pixels. Renderingsettings can be adjusted to minimise the obviousness of this effect to aviewer. The lens system may introduce effects due to chromaticaberration or other focal parameters of the lens system. This can bepre-corrected for by providing a distorted image on the screens, asdescribed in more detail below. In addition to the lens system causingdistortions, the display panels themselves may be curved or angledrelative to one another or to the sight axis of a viewer. This effect,particularly when filtered through a lens system as set out above, canrequire additional pre-rendering to provide the desired images to aviewer. Indeed, even flat lenses may be positioned a variable distancefrom the user's eyes (e.g. to improve user comfort), which may requirethe rendering parameters to be varied to account for this. Depending onthe design of the headset, ambient lighting may be taken into account,for example to increase brightness output by the panels if the ambientlighting is bright, so as to ensure that the images displayed arevisible.

In addition, the user may have various accessibility requirements whichaffect how messages are to be displayed. For example, the user may havevisual impairment, colour blindness, astigmatism, issues with the retinaor other eye parts, macular degeneration, etc., all of which may requireadjustments to the arrangement of images or image colours to bedisplayed, and which can be at least partly corrected for bypre-rendering accordingly. In some cases, the pre-rendering may includeenhancing brightness or saturation towards the centre of the displayedimage. In some cases, it can be helpful to think of the lens system asextending to include also prescription eyewear (glasses, contact lenses,etc.) that a user may be wearing. The user may adjust the displayparameters (and therefore the rendering parameters) to account for this.In some cases, the user may make an adjustment (with correspondingrendering parameter adjustments) to adapt the headgear to theirinter-pupillary distance.

There may be different rendering settings based on circumstances or userinformation. For example, information may be modified before beingdisplayed to a child or adult with learning difficulties. In anotherexample, where the display unit is being used to train a user (e.g. incounter terrorism, paramedic, firefighting, etc.), a flag may be set topresent slightly different information in the event of an error, toensure that the user is aware that the error message is not part of thetraining or simulation, but relates to a real-world event.

Additional parameters to take into account for rendering steps are onesrelated to user activity. For example, if it is known that the currentdisplay data is likely to require a large amount of movement from theuser, it may be desirable to alter how messages are displayed, e.g. butcausing them to slowly fade into view, since rapid changes in perceivedmovement (especially in VR situations) can cause nausea anddisorientation. In some cases, a user may lose their balance if there isa rapid transition from fast movement to no movement. Likewise, if thedata being presented is typically very dark, a bright message could bejarring and painful to a user. This could be predicted and adjusted for,for example by providing a dimmer message image the than the usualbrightness. In some circumstances a very bright average data may resultin a brighter than average message being displayed, for similar reasons.General information about the typical scene likely to be encountered maybe provided, for example by considering an entropic measure of the“busyness” of the background.

Optionally, the memory is a non-volatile memory. The use of non-volatilememory allows the correctly rendered messages to be stored in the memoryof the peripheral display unit in such a way that shutting down theperipheral display unit will not erase the correctly rendered messages,nor will accidental or deliberate powering down of the peripheraldisplay unit.

In some cases, the pre-rendered information messages are paired withperipheral display unit statuses and the peripheral display unit isconfigured so that, in the event that a particular status is detected,the corresponding pre-rendered information message is displayed withoutrequiring data transmission from the base station. This allows a user tobe correctly alerted to a wide variety of statuses even when there is noconnection available. Examples of such statuses and messages include:

-   -   No connection is available. This is a key case where it is        important that the message be provided in advance, since it will        not be possible to render this message correctly when the error        occurs.    -   Low bandwidth. For similar reasons as set out above, this is        important to have in advance.    -   No data detected. In the case where the connection is (or        appears to be) working correctly, but data is nonetheless not        being received, it will be impossible to render messages to        alert a user to this.    -   Base station and display unit are incompatible. In some cases,        it may be that these two parts of the system cannot communicate        with one another adequately to render the data required of them.        They may nevertheless have an emergency set of parameters which        can be used to provide a readable message to a user to alert        them to the problem.    -   System damage. Part or all of the system may suffer an error        (e.g. due to software updates or physical damage) which prevents        further communication.    -   Antennae misaligned. In this case, the connection quality may be        reduced (resulting in a specific case of the above errors).    -   Proximity/out of bounds. The user may be reaching the edge of        the usable region (in the real world), either due to        communication (e.g. line of sight) limitations or due to        physical space limitations (the room they are in is too small).    -   Overheating. One or more parts of the system may be about to        shut down or otherwise fail.    -   Fire/system alarm. Alert the user to events outside of the        virtual world.    -   Headphones not present.    -   Microphone muted.    -   Audio in/out interrupted.    -   Session expired/insert more money or tokens/unauthorised access.        For pay per hour systems etc. there may be a need to alert a        user to the situation when a host system is configured to shut        down after a certain time, etc.    -   Host system restarting/updating/busy. Where the host is required        to restart, update, or generally be busy doing something else,        the user should be able to be alerted, since the host may be        unresponsive at such times.    -   Power system failure/critical.    -   Fingerprint/voice recognition request. In case a user finds        themselves locked out of the host.    -   Too many devices on the same radio channel (channel congestion).        This would be a special case of the low bandwidth case above.

In any of the above cases, the message may include suggestions such as“move closer to base station” or “check for line-of-sight obstacles” totry to remedy the situation, in addition to alerting the user to thesituation. Audio channels may be used to back up or add to the messagessent. Audio streams could be sampled and sent to the device to be storedfor the above scenarios too. This could be used in situations where theimage should not be interrupted, or if the user has accessibilityissues, or if the display panel is reporting problems/is missing, or theevent is particularly critical, e.g. fire alarm.

The pre-rendered information messages may be error messages and thestatuses may be error statuses. This is particularly important as manyerrors may make it impossible for the peripheral display unit to contactthe base station (or indeed anything else). Consequently it isadvantageous to prepare in advance for such error statuses bypre-loading error messages to prompt a user to take corrective action.In particular when there is no signal available due to the unreliablelink, it is important to be able to inform a user of this, so that theycan move the peripheral display unit in an attempt to re-establish theconnection. It is important that such a message is correctly rendered asin extreme cases it may be impossible to read a poorly rendered message,leaving the user in no better of a situation than a blank screen wouldhave. Situations may even arise in which the peripheral display unitdoes not know what the solution is. In these cases, it may beadvantageous to have a default error message to revert to. This may bean instruction to move the display unit closer to the base station in anattempt to improve the communications link, so that the base station canbe interrogated (possibly even utilising a remote server, e.g. via aconnection to the internet, to help resolve the issue), for example.

In some cases the peripheral display unit is configured to send messagesto be rendered to the base unit in the event that availability of aconnection between the peripheral display unit and the base station isdetected. This allows a peripheral display unit to be provided with aset of messages which might be useful to it, i.e. to cover the varioussituations envisaged by the manufacturer. When a communications link isavailable, the peripheral display unit can send these messages to thebase station which performs operations on the basic images to renderthem for display by the peripheral display unit, before sending therendered images back to the peripheral display unit.

Additionally or alternatively, the base station may be configured todetermine messages for sending to the peripheral display unit. Thisallows the base station to provide the peripheral display unit withadditional information for display, which may not have been available tothe peripheral display unit at the time it was manufactured. Theperipheral display unit may be given the opportunity to accept or rejectmessage proposals. In the event that the messages are to be sent to theperipheral display unit, the rendering of the messages can occur in thesame manner as that set out above. The proposed messages may be pairedwith status conditions, as described above. In some cases, this pairingmay include additional, non-visible information for the peripheraldisplay unit, to tell it how to determine when the corresponding statushas occurred, for example.

The pre-rendering may take account of one or more of: language settingsof the base unit and/or the peripheral display unit; chromaticaberration of the peripheral display unit optics; focal parameters ofthe peripheral display unit optics; brightness variation in pixels (e.g.mura artefacts); other display parameters of the peripheral displayunit; and/or one or more parameters of image or video data currentlybeing displayed by the peripheral display unit. Each of these isimportant for ensuring that the message can be seen and correctlyinterpreted by a user. For example if a user only speaks English, thenmessages in Chinese will be of no use to them. A user may have set theirlanguage preferences on the base station, so there is no way that theperipheral display unit would know which language to present themessages in without communicating with the base station. Similarly,different peripheral display units each of which are compatible with thebase station may nonetheless require different rendering to account fordifferences in their optical systems and/or user preferences. When theperipheral display unit connects to the base station, part of thehandshaking and verification stage can include the peripheral displayunit transmitting some of these parameters to the base station to allowthe base station to correctly render the images or video. Lastly, it ispossible that the information message to be overlaid on the display mayneed to be adapted to the other video data being sent. For example, inorder to read text on a background, the text should contrast with thebackground, so the message rendering may take into account average pixelcolours, most common pixel colours (or colour ranges) averagebrightness, etc. to maximise comfort or minimise discomfort from jarringcolour combinations or brightness transitions. In addition, thecorrections required to correctly display information to a user toaccount for curved or angled panels, distance of a display panel from aviewer's eyes, ambient light levels, accessibility requirements,inter-pupillary distance, user or circumstantial factors, and/or motionor location of a user.

The system may be configured to send pre-rendered information messagesto the peripheral display unit every time a connection is available.This ensures that the most up to date settings are used.

Additionally or alternatively, the system may be configured to sendpre-rendered information messages to the peripheral display unit everytime the base station and/or the peripheral display unit is initialised.In other words, when the base station and/or peripheral display unit isswitched on and loaded up (i.e. booted), there may be additionalmessages sent to supplement the existing messages or replace the entireset. This provides a good periodic update to the system, and also allowsthe message display settings to be reset to default values in the eventthat any display parameters of the system are also reset to theirdefaults.

Additionally or alternatively, the system may be configured to sendpre-rendered information messages to the peripheral display unit aftersoftware for the base station and/or the peripheral display unit hasbeen updated. This allows the messages to be correctly rendered in theevent that any relevant settings are changed in a software update. Thisis particularly important as error statuses may sometimes be more likelyjust after a software update.

Additionally or alternatively, the system may be configured to sendpre-rendered information messages to the peripheral display unit after alanguage setting for the base station and/or the peripheral display unithas been changed. As discussed above, the language in which a message ispresented can be critical to prompting a user to take the desiredaction. Similarly, the system may be configured to send pre-renderedinformation messages to the peripheral display unit after a graphicaldisplay setting has been changed. As set out above, the graphicaldisplay settings can be key to getting the message to display in thedesired manner to allow it to be correctly read by a user. In AR and VR,this is particularly important, since out of focus or otherwise poorlyrendered images are known to cause discomfort and nausea in users.

Additionally or alternatively, the system may be configured to sendpre-rendered information messages to the peripheral display unit toupdate the pre-rendered information messages stored on the peripheraldisplay unit in the event that a change in one or more parameters ofimage or video data currently being displayed by the peripheral displayunit is detected. The parameters may relate to average brightness orcolour parameters, for example. This can allow messages to be providedwith the most up to date information about the makeup of a typicalvisual frame. Consequently, an appropriate level of contrast andnon-jarring colour and brightness combinations can be ensured.

Additionally or alternatively, the system may be configured to sendpre-rendered information messages to the peripheral display unit inresponse to detection of a particular status by the base station and/orthe peripheral display unit. This allows, for example, the peripheraldisplay unit to detect that it has accidentally deleted or overwrittenone or all of the messages, and to request that it be sent replacements.

Additionally or alternatively, the system may be configured toperiodically send pre-rendered information messages to the peripheraldisplay unit. This can ensure that the messages are kept up to date,even if none of the other triggering events discussed above occur.

In each of the above examples in which the system is configured to sendpre-rendered messages based on some trigger, the new pre-renderedmessages which are sent will also be stored in the peripheral displayunit memory. In some cases, these will overwrite the previous equivalentmessages. In other cases, they may be stored in addition to the existingmessages. Clearly, this latter case will eventually fill the availablestorage space. Consequently, the peripheral display unit may delete theoldest messages to make way for the new. Alternatively it may delete theleast used messages to make space for the new messages.

The method and/or system described above may also include features whichdecide how to deal with sending the pre-rendered information messageswhen other data is also being sent. In this case, the other data maycomprise video and/or audio data, for example, for display by theperipheral display device. The pre-rendered information messages may besent in preference to other data. For example, the system may decidethat the sending of some or all of the messages is more important thanthe sending of other data. This may occur when the link is veryunreliable and the system urgently needs to alert a user of this fact,for example, so that the user can address the issue quickly. In anycase, since such messages may form only a single frame (or indeed only aportion of a frame), the messages can be prioritised and only the mostimportant messages be sent by dropping relatively few frames inpreference to sending the messages.

In some cases, the messages are sent concurrently with data. Forexample, the messages can be packaged with the data. In this case, aheader associated with the data and/or the messages may be modified toalert the peripheral display unit to the presence of the messages. Thisprovides a convenient manner in which the sending of messages can beincluded in the normal data stream, requiring only minimal modificationof the existing data transfer protocols.

Additionally or alternatively, the method may include detecting thebandwidth availability of the connection once availability of aconnection between the peripheral display unit and the base station isdetected. In the event that the available bandwidth of the connection isinsufficient for both the messages and the data, bandwidth allocated forsending the messages may be reduced. Additionally or alternatively, inthe event that the available bandwidth of the connection is insufficientfor both the messages and the data, sending of the messages may bedelayed until the detected bandwidth is larger than a predeterminedthreshold. Additionally or alternatively, in the event that theavailable bandwidth of the connection is insufficient for both themessages and the data, bandwidth allocated for sending the data may bereduced.

The above additional method steps, while loosely presented as methodsteps, can also form part of the system. For example the systemdescribed herein can be configured to perform these method steps, inaddition to some or all of the functions of the system describedelsewhere.

Additionally, in each of the above examples, while the bulk of therendering is performed by the base station, some of the examples may bebenefitted by the peripheral display unit performing some of therendering. For example, where the message to be displayed is provided asa transparent overlay, the exact background image will be an importantfactor in displaying the message as intended. Merging the message withthe background is an example of rendering which may be performed atleast in part at the peripheral display unit. Similarly, where thingslike the brightness or busyness of the background is factored into thedisplayed message, a degree of local rendering at the peripheral displayunit may be beneficial in displaying the image as intended.

To assist in this, the peripheral display unit may have the ability todetermine locally the busyness, brightness or other parameters of theimages being displayed, and to use this information in the process setout above. For example, these measured parameters may be supplied to thebase station for factoring into the rendering performed there, or theperipheral display unit may use them to locally adjust the pre-renderedimages. Lastly, the display unit may use the measured parameters toselect from a range of pre-rendered images (e.g. a dark or a lightversion to improve the contrast, for example).

BRIEF DESCRIPTION OF THE DRAWINGS

Specific embodiments of the invention will now be described withreference to the Figures, in which:

FIG. 1A shows a schematic of a virtual reality or augmented realityheadset visual display unit;

FIG. 1B shows a schematic of a smart phone visual display unit;

FIG. 1C shows a schematic of a projector visual display unit;

FIG. 2A shows a schematic of a base station and visual display unit;

FIG. 2B shows a schematic of an alternative arrangement of a basestation and a visual display unit; and

FIG. 3 shows a flow chart for implementing a method disclosed herein.

DETAILED DESCRIPTION

Turning now to FIG. 1A, which shows a visual display unit in the form ofa virtual reality (VR) or augmented reality (AR) headset 104, asillustrated in FIG. 2, connected to a host device 102, which may be acomputing device, gaming station, etc. and is an example of a basestation from which signals and data are sent. The virtual realityheadset 104 incorporates two display panels 106, 108, which may beembodied as a single panel split by optical elements. In use, onedisplay is presented to each of a viewer's eyes. The host device 102generates image data for display on these panels 106, 108 and transmitsthe image data to the virtual or augmented reality headset 104.

Augmented reality glasses also have two display panels, each associatedwith one of the user's eyes, similar to the virtual reality headset 104shown in FIG. 1A. This example differs in that the display panels aretranslucent so as to allow a user to view their environment through thepanels 106, 108 as normal when no signal is sent. When signal is sent,this is overlaid on the user's view of their environment, modifying thatview by adding additional visual information to the user's view, therebyaugmenting what would have been seen without the signal being sent.

The host device 102 may be a static computing device such as a computer,gaming console, etc., or may be a mobile computing device such as asmartphone or smartwatch. As previously described, the host device 102generates or receives image data and transmits the data to the augmentedreality glasses or virtual reality headset 104 for display.

The display device may be connected to the host device 102 by a wired orwireless connection. While a wired connection minimises latency intransmission of data from the host to the display, wireless connectionsgive the user much greater freedom of movement within range of thewireless connection and are therefore preferable. A balance must bestruck between high compression of data, in particular video data, whichcan be used to enable larger amounts of data (e.g. higher resolutionvideo) to be transmitted between the host and display, and the latencythat will be introduced by processing of the data. In either the wiredor the wireless case, the link may be an unreliable link, in which thereis a higher than normal incidence of packets or frames being dropped,simply not arriving having been sent, noise interfering with the signal,random bit-flipping, etc.

Ideally, the end-to-end latency between sensing a user's head movement,generating the pixels in the next frame of the VR (virtual reality)scene and streaming the video should be kept below 20 ms, preferablybelow 10 ms, further preferably below 5 ms.

The wireless link should be implemented as a high bandwidth short-rangewireless link, for example at least 1 Gbps, preferably at least 2 Gbps,preferably at least 3 Gbps. An “extremely high frequency (EHF)” radioconnection, such as a 60 GHz radio connection is suitable for providingsuch high-bandwidth connections over short-range links. Such a radioconnection can implement the WiFi standard IEEE 802.11ad. The 71-76,81-86 and 92-95 GHz bands may also be used in some implementations.

The wireless links described above can provide transmission between thehost and the display of more than 50 frames per second, preferably morethan 60 fps, further preferably more than 90 fps. In some very recentexamples, frame rates of 120 fps are being used.

In each of the VR and AR embodiments described above, the aim is to makethe user feel immersed in the virtual/augmented environment. Part ofthis includes providing a lens system between the display panels 106,108 and a user's eyes to give a feel of a focal depth a few metres fromthe user (rather than the actual distance of a few centimetres). Thisextreme change in depth requires strong lenses to be used, and the useof strong lenses in turn enhances various optical aberrations due to thelens system. In order for images or video (henceforth referred to simplyas “images”, with the understanding that “images” should be taken toinclude video as well) to be displayed at the desired depth and freefrom aberrations, the raw images displayed on the display panels 106,108 are themselves distorted. This distortion is calculated in orderthat the various optical aberrations introduced by the optical systembring the images back to a desired, undistorted image. The calculationof the correct distortion to apply to the image to correctly displaythat image is known as rendering (or pre-rendering), and takes as aninput raw image data for display and the parameters of the visualdisplay unit (e.g. lens types, strengths, etc.), and outputs a distorted(pre-rendered) image. As such rendering is computationally demanding itis usually performed at a base station (e.g. host device 102). Images inVR/AR systems are usually pre-rendered (that is, supplied to theperipheral display unit already rendered), e.g. by the base station 102.

As will be described in more detail later, the base station 102 and/orthe headset may need to communicate statuses directly to a user fromtime to time. It is important that a user is able to correctly read suchstatuses with minimal effort, which requires (among other things) thatthe statuses be presented in a language with which the user is familiar,and that the messages is correctly rendered for display. Moreover, theheadset 104 (visual display unit) should be able to display the statusmessages whether or not it is in communication with the base station102, since a particularly useful message class may relate to issues withthe connection between the base station 102 and the visual display unit104. For example, it is useful to alert a user that the connection hasfailed, but in the case of a failed connection, it would be impossibleto render such a message correctly. Other messages can also be providedas set out below, which are not strictly dependent on the connection (orlack of), which has the benefit that: (1) all messages are treated thesame, thereby simplifying protocols; (2) messages unrelated to theconnection health can still be displayed when needed in the event thatthe occurrence of their corresponding status happens to coincide with aconnection failure; and (3) sporadic sending and receiving ofunnecessary data over the link is reduced, thereby reducing burdening ofthe connection and the rendering module since all messages are sent atthe same time, rather than in a piecemeal fashion. The headset 104 maybe configured to detect user-selected focal parameters of the opticalsystem in the headset and communicate these (via connection 110) to thebase station 102. These parameters are then used to ensure that therendering is performed correctly so that the images are correctlydisplayed to a user, in some cases in combination with other parameterssuch as those related to the operation of the base station 102.

FIG. 1B shows a system which is similar in operation to the embodimentshown in FIG. 1A. In this case, however, there is no separate hostdevice 102. The entire system is contained in a single casing 112, forexample in a smartphone or other such mobile computing device. Thedevice contains a processor 116, which generates display data fordisplay on the integral display panel or screen 104, which can bethought of as a form of peripheral display unit.

In this case, the data for display may be provided from a remote sourcevia a wireless communication 110, e.g. wireless internet or mobiletelephony. Such data is received via antenna 114, which is subsequentlyfed to the processor. In the event that this wireless connection fails,a notification from the content provider may be displayed, consistentwith the method set out below, correctly rendered and being as up todate as possible by using the method disclosed herein.

The mobile computing device may be mounted such that the screen 104 isheld in front of the user's eyes as if it were the screen of a virtualreality headset. In some cases the display panel 104 is artificiallysplit using a blank line of a single colour (e.g. black). Moreover, thescreen may be held in place using a special frame or holder. The holdermay include a central partition to prevent light from the right half ofthe screen 104 from entering the left eye and vice versa.

Moreover, the holder may include a lens system similar to that describedabove in relation to the VR/AR headsets. In this case, careful renderingof the images is important to ensure that the user sees the images inthe manner intended.

FIG. 1C shows yet a further example, in which the peripheral displayunit is in the form of a projector 104. As in the previous examples, theprojector 104 receives information for display from a base station 102,for example a host device such as a computer, gaming console, etc. Oncemore it is possible that the base station 102 is connected to a remotelocation e.g. via the internet or other network which supplies the datafor display. Alternatively, the displayed data may be generated by thebase station 102 itself, for example from installed software or a DVD orCD. In any event, the data for display is sent to the projector 104 viaa communication link 110. The projector 104 projects the visual data fordisplay onto a portion 122 of a suitable surface 120 (a special screen,or even a blank white portion of wall, for example), via projectionoptics 118.

The projection optics 118 create a similar problem to that set out abovein respect of the VR/AR headset, specifically that it is typicallydesired that the image formed on the screen or wall is usually intendedto be rectilinear. Depending on the arrangement in place, the projector104 may not be positioned such that the optical axis is notperpendicular to the plane of the portion of the wall 122. Consequently,not only must the projection optics 118 bring the images into focus atthe correct distance (i.e. the distance between the projector 104 andthe wall 120), but must also adapt to the non-perpendicular arrangementbetween the optical axis and the surface 120. Finally, there will beportions, e.g. towards the edges of the image, which travel a furtherdistance than those portions towards the centre of the image. Thiscauses the image at those edge portions to spread out more, and thus todistort the preferred rectilinear shape of the displayed image. Theprojection optics 118 can be configured to correct this effect andensure that the images are displayed in focus and in the desired sizeand shape.

As set out above the use of optical systems, while allowing the imagesto be presented in the desired format, can introduce other aberrationswhich affect the image quality. Consequently, the images sent to theprojector 104 may be pre-rendered by the base station 102 (or even by aremote source which feeds the base station) so that by the time theimages arrive at the portion of the surface 122, the additionaldistortion caused by the projection optics 118 cancels out thepre-rendering applied to the images and the images are displayedcorrectly, i.e. without distortion or aberration. The projector 104 maybe configured to detect user-selected focal parameters of the projectionoptics 118 and communicate these (via connection 110) to the basestation 102. These parameters can then be used to ensure that therendering is performed correctly and that the images are correctlydisplayed.

In the event that it is necessary to display error or informationmessages, these too should be pre-rendered so that a user can correctlyinterpret the information being provided. As set out above (and will bedescribed in more detail below), the base station 102 can be used toensure that the information/error messages are correctly rendered, byusing the optical parameters in much the same manner as it does toensure that the visual data for display are correctly pre-rendered.

FIG. 2A shows a block diagram overview of a system according to thepresent disclosure, and provides a little more detail on how the systemsin FIGS. 1A to 1C operate. A host computer 202 (e.g. a base station) isconnected to a display control device 224, which is in turn connected toa display device 226. Collectively, display control device 224 and thedisplay device 226 form a peripheral display unit, for example projector104 or headset 104 described in detail above. The host 202 contains anapplication 228, which produces display data, by reading from a disc oronboard storage, receiving from an internet or network source, etc. Atthe same time as it is produced (or received), the data may also bepre-rendered by the application 228, consistently with the meaning ofpre-rendering set out above. The display data may be produced and sentfor compression either as complete frames or as canvasses, which may,for example, be separate application windows. In either case, they aremade up of tiles of pixels, where each tile is a geometrically-shapedcollection of one or more pixels.

The display data is sent to a compression engine 230, which may comprisesoftware running in a processor or an appropriate hardware engine. Thecompression engine 230 first performs an encoding of the data, forexample using a Haar transformation, to convert the data into a formatthat may then be further compressed, minimising data loss.

The compression engine 230 may then further compress the data andthereafter sends the compressed data to an output engine 232. The outputengine 232 manages the connection 210 with the display control device224 and in this example includes a socket for a cable to be plugged intofor a wired connection to a corresponding input engine 236 on thedisplay control device 224. The compression stage is useful because,since the context is one in which a high bandwidth between the basestation 202 and the peripheral display unit 224, 226 cannot beguaranteed, minimising the amount of data to be sent can greatly improvethe robustness of the system. In some examples, e.g. the self-containedsystem of FIG. 1B, there may be no need for compression/decompressionwithin the self-contained unit, since the internal connections can bewired, meaning that there is less of an issue with bandwidth.

The connection 210 between the output engine 232 and the input engine236 may include an unreliable portion 234 (or indeed the entireconnection may be unreliable). This may happen in the form of an IPnetwork having variable traffic, and consequently introducing a variable(and on occasion excessive) signal delay. Alternatively, the network mayroute via different routes which themselves have differentbandwidths/delay times which can cause delays in data arriving,particularly when e.g. video frames are split into many parts beforetransmission and each part takes a different route and arrives at adifferent time. In other examples, the unreliable link may occur formore prosaic reasons; the cable of the connection may be physicallydamaged or noise may enter the system by way of RF pickup or otherinterference.

In another example, shown in FIG. 2B, the system operates in much thesame manner, with identical numbers representing features with the samegeneral functionality. An application 228 is connected to a compressionengine 230, which is subsequently connected to an output engine 232, allof which form part of the base station 202. A connection 210 to theperipheral display unit 224, 226 is provided which results in data beingreceived at an input engine 236, which is subsequently fed to adecompression engine 238, then a scaler 240 and another output engine242. The output engine 242 sends the data stream to the display device226.

The difference between the system in FIG. 2A and the system in FIG. 2Bis that the connection in FIG. 2A is a wired connection, while in FIG.2B the connection 210 is a radio transmitter for a wireless connection.As shown, the output engine 232 is provided with a wireless communicator244 for transmitting and/or receiving data to/from the peripheraldisplay unit 224, 226 via electromagnetic radiation. Similarly, inputengine 236 is provided with a wireless communicator 246 configured totransmit and receive signals to/from the base station 202.

Wireless communications can be unreliable for a number of reasons.Interference from other electromagnetic sources, movement of the sourceand/or receiver (to block the signal, move out of range, or move farfrom the axis a beam formed for the transmission of signals), or thepresence of other objects (e.g. people) moving and blocking the signalcan all result in a reduced bandwidth available, or indeed in the linkfailing entirely.

In any case, the data, once it arrives, is fed into the input engine236, which is itself connected to a decompression engine 238. When itreceives compressed data it sends it to the decompression engine 238 orto a memory from which the decompression engine 238 can fetch itaccording to the operation of a decompression algorithm. In any case,the decompression engine 238 may decompress the data, if necessary, andperforms a decoding operation, optionally using a reverse Haartransform. In the illustrated system, the decompressed data is then sentto a scaler 240. In the case where the display data was produced andcompressed as multiple canvasses, it may be composed into a frame atthis point.

If scaling is necessary, it is preferably for it to be carried out on adisplay control device 224 as this minimises the volume of data to betransmitted from the host 202 to the display control device 224, and thescaler 240 operates to convert the received display data to the correctdimensions for display on the display device 226. In some embodiments,the scaler may be omitted or may be implemented as part of thedecompression engine. The data is then sent to an output engine 242 fortransmission to the display device 226. This may include, for example,converting the display data to a display-specific format such as VGA,HDMI, etc.

In some cases, the connection 210 comprises both wired and wirelessportions such as those shown in FIGS. 2A and 2B.

In order to provide pre-rendered information messages to the peripheraldisplay unit 224, 226, the messages must first be rendered to accountfor the display settings currently in use. This rendering can beperformed in the same manner as the pre-rendering for the visual data tobe displayed by the peripheral display unit 224, 226. For example, theapplication 228 may perform the rendering for the messages so that theycan be correctly displayed. These can be transmitted from the basestation 202 to the display control device 224 via wired or wirelessconnection 210. Once the pre-rendered information messages are receivedby the display control device 224, they are stored in a memory 248 untilneeded. For example, a particular status is detected (by the peripheraldisplay unit 224, 226 and/or the base station 202) the display controldevice may be triggered to display the corresponding information/errormessage. This is displayed by the peripheral display unit 224, 226,irrespective of the presence or absence of the connection 210.

In each of the examples of FIGS. 2A and 2B, the display unit 224, 226may comprise a VR adapter (e.g. unit 224) and a separate display device226. This arrangement provides a separation between the signal receivingand decompression parts (224) and the display parts 226. A user can, forexample, connect different headsets 226 to the same VR adapter 224. Thisallows a user to upgrade their headset, for example when a newer,higher-resolution version comes out, without needing to update otherpieces of hardware. The present invention is particularly useful in thiscase, since the headset need not be integral to the VR adapter, therebyincreasing the number of variables in the rendering process. Moreover,each headset may store its own pre-rendered messages locally, meaningthat error messages (e.g. base station is expecting a different headset)can be displayed without the base station's involvement.

In particular, this sending of the correctly rendered messages may notbe possible if the connection has failed (due to the various reasons forunreliability set out above, for example). Therefore, the systemaddresses this using the method shown in FIG. 3. This method begins atstep 350 by detecting that a connection (e.g. connection 210) is presentbetween a peripheral display unit and a base station. In the case of thesystem shown in FIG. 1B, the connection could be the wireless connection110 through which data is streamed, and the base station could be aremote part of the mobile network, for example a content server. Themessages for rendering and sending to the peripheral display unit may beprovided originally (in a non-rendered format) from the peripheraldisplay unit. That is the peripheral display unit may decide whichmessages it needs to have rendered, and then request that these berendered for it to display. Additionally, or alternatively, the basestation may propose messages for rendering and storage, for example toprovide information for display in response to a specific base stationstatus, or if the base station is in communication with a manufacturervia the internet new messages may be provided to the peripheral displayunit.

Once the connection has been detected, in step 352 the system sendspre-rendered information messages from the base station to theperipheral display unit.

Finally, in step 354, the system stored the pre-rendered informationmessages at the peripheral display unit. These messages can then bedisplayed in response to a status detected at the peripheral displayunit. The storage at the peripheral display unit may comprise a memory,for example a non-volatile memory.

The invention claimed is:
 1. A method for displaying informationmessages by a peripheral display unit, comprising: projecting asimulated environment on a display device associated with the peripheraldisplay unit; receiving one or more pre-rendered information messagesassociated with one or more events, respectively, outside the simulatedenvironment; storing the one or more pre-rendered information messages;detecting an occurrence of at least one of the one or more events; andprojecting at least one of the one or more pre-rendered informationmessages on the display device responsive to detecting the occurrence ofat least one event.
 2. The method of claim 1, wherein the one or morepre-rendered information messages are received from a base stationconfigured to pre-render one or more input images to produce the one ormore pre-rendered information messages, and wherein each of the one ormore pre-rendered information messages is pre-rendered to be displayedas virtual reality (VR) or augmented reality (AR) content.
 3. The methodof claim 2, wherein the pre-rendering is based at least in part on: alanguage setting of the base station; a language setting of theperipheral display unit one or more optical effects caused by one ormore focal parameters of a lens system within the peripheral displayunit; mura artefact effect; alignment of a display screen associatedwith the peripheral display unit in relation to a sight axis of a userof the peripheral display unit; a physical condition of the user's eyes;or a plurality of pixel values associated with an image currently beingdisplayed or anticipated to be displayed by the peripheral display unit.4. The method of claim 2, further comprising updating the stored one ormore pre-rendered information messages in response to: the base stationbeing initialised; the peripheral display unit being initialised;software for the base station being updated; software for the peripheraldisplay unit being updated; a language setting for the base stationbeing changed; a language setting for the peripheral display unit beingchanged; a graphical display setting being changed; or a change in aplurality of pixel values associated with an image currently beingdisplayed by the peripheral display unit.
 5. The method of claim 1,wherein the one or more pre-rendered information messages are stored ina non-volatile memory at the peripheral display unit.
 6. The method ofclaim 1, wherein the one or more events include lack of connection, lowbandwidth, loss of data, system damages, limitation in proximity, orfire alarms.
 7. The method of claim 1, wherein the method is executedperiodically.
 8. A system for displaying one or more pre-renderedinformation messages, the system comprising: a base station; aperipheral display unit having a memory, wherein the peripheral displayunit is configured to project a simulated environment on a displaydevice associated with the peripheral display unit; receive one or morepre-rendered information messages associated with one or more events,respectively, outside the simulated environment; store the one or morepre-rendered information messages in the memory; detect an occurrence ofat least one of the one or more events; and project at least one of theone or more pre-rendered information messages on the display deviceresponsive to the detected occurrence of at least one event.
 9. Thesystem of claim 8, wherein the base station is configured to pre-renderone or more input images to produce the one or more pre-renderedinformation messages, wherein each of the one or more pre-renderedinformation messages is pre-rendered to be displayed as virtual reality(VR) or augmented reality (AR) content.
 10. The system of claim 9,wherein the base station is configured to pre-render the one or moreinput images to produce the one or more pre-rendered informationmessages based at least in part on: a language setting of the basestation; a language setting of the peripheral display unit; one or moreoptical effects caused by one or more focal parameters of a lens systemwithin the peripheral display unit; mura artefact effect; alignment of adisplay screen associated with the peripheral display unit in relationto a sight axis of a user of the peripheral display unit; a physicalcondition of the user's eyes; or a plurality of pixel values associatedwith an image currently being displayed or anticipated to be displayedby the peripheral display unit.
 11. The system of claim 8, wherein theone or more events include lack of connection, low bandwidth, loss ofdata, system damages, limitation in proximity, or fire alarms.
 12. Thesystem of claim 8, wherein the peripheral display unit is configured toupdate the stored one or more pre-rendered information messages inresponse to: the base station being initialised; the peripheral displayunit being initialised; software for the base station being updated;software for the peripheral display unit being updated; a languagesetting for the base station being changed; a language setting for theperipheral display unit being changed; a graphical display setting beingchanged; or a change in a plurality of pixel values associated with animage currently being displayed by the peripheral display unit.
 13. Thesystem of claim 8, wherein the peripheral display unit receives the oneor more pre-rendered information messages periodically.
 14. The systemof claim 8, wherein the system is configured to update the one or morepre-rendered information messages stored in the memory of the peripheraldisplay unit in response to a change in the detected occurrence of atleast one event.